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ABSTRACT 


Decision makers are often confronted with problems for which there 
exist several distinct measures of success. Such problems can often 
be expressed in terms of linear or nonlinear programming models with 
several "criterion" functions instead of single objective functions. 
A variety of techniques have been applied to multicriterion problems, 
but the approach used here, "The Dynamic Programming Approach to Multi- 
criterion Optimization Problem," is based on the concept that the ideal 
solution to a multiobjective problem must be a pareto optimal solution. 
In many cases simply narrowing the set of candidate solutions to the 
set of all pareto optimal solutions may enable the decision maker to 
find the compromise being sought. The determination of nondominated 
points and corresponding nondominated values (pareto optimal solution) 
related to the multicriterion optimization problem is approached through 
the use of dynamic programming. The dynamic programming approach has an 
attractive property which provides the basis for generation of non- 
dominated solutions at each stage by the decomposition method. By 
using recursive equations we can find out the nondominated points and 


corresponding nondominated solutions of multiaggregate return function. 
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I. INTRODUCTION 


A. PROLOGUE 


Situations requiring a decision typically involve multiple conflict- 
ing goals which cannot be satisfied simultaneously due either to the 
inherent conflict among some goals (e.g., maximizing comforts in 
traveling between two points vs. minimizing cost of travel. Maximizing 
effectiveness vs. minimizing cost) or to a scarcity of resource (e.g., 
maximizing productions on each of several individual production lines). 

Since first described by Pareto many years ago, the problem of 
multicriteria, or vector valued optimization has fascinated economists, 
planners and engineers. Although it has also been considered in a very 
abstract framework by a number of researchers the most commonly en- 


countered form of the problem seems to be as follows: 


Maximize [£, (x) ; £. (x) ae £. (x) | 


subject to g, (x) SS Up. Sede Sorin 


where x 1s a vector. 

During the past 10 years increased research in this area has been 
directed in several directions reflecting different classes of multiple 
objective decision problems. An excellent recent summary of this area 
is by K. R. MacCrimmon, [14]. Roy [15] summarized several approaches in 
1971 and Johnsen [16] offers an almost encyclopedic coverage of the 


field to 1975. 





A lot of researchers solved the MCO problem by using the concept of 
a nondominated solution. Yu [17] introduced a general concept of domi- 
nation structures; the idea helps to view the one-dimensional comparison 
and the set of nondominated (pareto optimal) solutions as two special 
cases of a more general solution concept. Zeleny [18 | develops a multi- 
criteria simplex method for generating all nondominated extreme points. 
Polak [19 ] presents an algorithm which constructs an economical grid of 
noninferior points to be used in computation with an interpolation 
scheme in the value space. The algorithm 1S specifically designed for 
the bicriteria case, but it can also be used in higher dimensional 
situations. 

The algorithm we shall present in this thesis is specialized to the 
two Or more criteria space with only a few equality or inequality type 
constraints (only one constraint in our example). 

The theory of the dynamic programming approach to the multiple cri- 
teria optimization problem [McopP ] is developed by combining the non- 
dominated solution concepts and dynamic programming method. 

Using the dynamic programming decomposition theorem, we break the 
multicriteria problem into N stages. At each stage we find the non- 
dominated solution by a recursion equation and hence solve the MCO 
problem by finding the nondominated solutions of multi-aggregate return 


function. 





B. MULTICRITERION OPTIMIZATION PROBLEM ANALYSIS - 
A REVIEW OF THE LITERATURE 
Maximize F(x) 


subject to g; (x) = Cm cor L=b,2,...,%0 


where x is an N-vector (x), Xoreee eX) and F(x), a function of x, is 
an L-vector with scalar components (£) (x), Ea(xX)y-- ee £, (x)). The 
solution to this problem is the set of all x which are nondominated and 
which satisfy the constraints g; (x) <0 for i=1,2,...,m. 

A solution vector x is nondominated if it satisfied the constraints 
and if there does not exist another x' also satisfying them and having 


the properties that 


£ s(x") < £, (x) Ors Valo? ooo, 


£ s(x") > £, (x) for at least one such i. 


—_ 


Although interest in multicriterion problems is quite recent, the liter- 
ature concerning these problems is already voluminous. 

For a bibliography and synthesis of work in this area see 
MacCrimmon fe) - In his paper, written in 1973, he discussed every 
method which was developed up to then and attempted to group them into 
four main categories to facilitate comparisons among the various methods. 
They are 

a) weighting methods 

b) seguential elimination methods 

c) mathematical programming methods 


d) spatial proximity methods. 





The class of weighting methods has received the most attention and 
particular models within this class have been the most widely applied. 
Although these weighting methods seem very diverse, they all have the 
following characteristics: 

- A process comparing attributes by obtaining numerical scalings of 
attribute values (intra-attribute preference) and numerical 
weights across attributes (inter-attribute preference). 

MacCrimmon divided the weighting method into nine weighting methods. 

Some of them are: 

linear regression 

trade-offs 

Simple additive weighting. 
The linear regression method can be used when the problem is in such 
Situations that are sufficiently repetitive to group together. Attri- 
butes considered by the decision maker can be treated as variables in 
a linear model. From his past choice, coefficients of the attributes 
are estimated using standard regression techniques. 

Slovic and S. Lichtenstein [3] provide a very comprehensive survey 
of linear regression models and have an excellent bibliography. 

The trade-off methods differ from the linear-regression method in 
that the preference of the decision maker is obtained by directly asking 
him his preferences rather than by inferring them from his past choices. 
Although this method has the advantage of obviating the need for a con- 
Siderable past history of similar situations, it has the disadvantage 
of possibly finding that the decision maker is unable to verbalize his 


true preferences. 





In the trade-off approach, the marginal rates at which the decision 
maker is willing to trade one attribute for another are obtained by 
direct questioning. If all the relevant trade-offs can be made, then 
the focus of the problem can be narrowed to one attribute. Raiffa [5] 
utilizes a more general trade-off method on the very complex problem of 
obtaining certainty equivalents for lotteries in multiple attribute 
decision problems. 

In the simple additive weighting method, for each of the attributes 
decision problems. 

In the simple additive weighting method, for each of the attributes 
the decision maker assigns importance weights which become the coef- 
ficients of the variables. To reflect his marginal worth assessments 
within attributes, the decision maker also makes a numerical scaling of 
intra-attribute values. He can then obtain a total score for each 
alternative simply by multiplying the scale rating for each attribute 
value by the importance weight assigned to the attribute and then summing 
these products over all attributes. 

After the total scores are computed for each alternative, the alter- 
native with the highest score is the one prescribed to the decision 
maker. Simple additive weighting is very widely used. One important 
use of this technique was in the mustering out system of the U. S. Army 
at the end of World War II [6]. 

Sequential elimination methods are less demanding of the decision 
maker than weighting methods. Sequential elimination methods are 
characterized by a process for sequentially comparing alternatives on 
the basis of attribute values so that alternatives can be either elimi- 


nated or retained. 


LO 





MacCrimmon distinguished among four methods. One of these is the 
dominance method which compares one real alternative against another 
real one to see if proper alternatives can be eliminated. If one 
alternative has attribute values that are at least as good as those 
of another alternative for all attributes, and if it has one or more 
values that are better, then the first alternative is said to “dominate" 
the second, and the second alternative can be eliminated. Although this 
is probably the least controversial decision rule, it unfortunately does 
not often succeed in eliminating very many alternatives. 

Terry [7] shows how dominance can be used as an initial filter in 
choosing among new product areas for comparing diversification. Freiner 
and Yu [8] combine dominance with the mathematical programming approaches. 

The class of programming methods has recently begun to receive much 
attention. MacCrimmon divided the mathematical programming methods into 
three methods, linear programming, goal programming and interactive, 
multi-criterion programming. 

A regular linear programming model may be viewed as a multiple 
attribute decision method. The variables are the attributes. The 
linear constraints are conjunctive constraints on combinations of 
attributes, and there is a linear, compensatory objective function. 

Efficient solution algorithms are available. The classic diet 
problem [9] is an example of this method. The objective is to provide 
a balanced diet, through the selection of the amounts of particular 
foods, that will satisfy particular nutritional standards at minimum 
cost. 

In a goal programming formulation, the decision maker specifies 


acceptable or desired levels on single attribute values (i.e., one- 


it 





variable constraints) or on combinations of attributes (i.e., multi- 
variable constraints) and these serve as the primary goals. 

Lee and Claytor [10] apply goal programming to the scheduling of an 
academic department. The attributes (variables) are defined in terms 
of faculty and non-faculty personnel having varying degree qualifica- 
tions. The goals involve faculty/student ratios, faculty/staff ratios, 
percentage increase in salaries, low cost, etc. Solutions obtained 
under different assumptions about available budget and accreditation 
requirements are presented. 

Ijiri [11] develops the accounting aspects of goal programming. 

Interactive, multi-criterion programming does not assume a global 
objective but rather requires the decision maker to provide his local 
trade-offs in the neighborhood of a feasible alternative. These trade- 
offs (on the attributes or criterion involving attributes) are used in 
a local objective function for a mathematical programming algorithm to 
generate an optimal solution for that objective. The decision maker 
then has an opportunity to provide new trade-offs which again serve as 
inputs to the algorithm. This process continues until the decision 
maker no longer wishes to revise his trade-offs and so an optimal 
solution is reached. 

Geoffrion, Dyer, and Feinberg [12] describe the use of the method 
in scheduling an academic department. Related approaches are given in 
the studies of Benayoun de Montgolfier and Tergny [13] and Yu, Zeleny. 

While the preceding three classes are probably the most common 
general approaches to multi-attribute decision problems, some of the 
more specialized methods are also receiving attention. Since they all 


make explicit use of spatial representation, MacCrimmon calls them 


1 





"spatial proximity methods." He distinguished among three methods in 
spatial proximity methods. One of them is the indfiference map method. 
The decision maker's preferences can be obtained in the form of 
indifference surfaces which show the combinations of attribute values 
that are equally preferred. The alternatives to be considered can be 

located in the same spatial representation, and by identifying the 
indifference surface on which they lie, a complete ordering among the 
alternatives can be generated. 

MacCrimmon [14] gives an example of the use of this procedure for 
transportation system planning. Many combined approaches are possible 
and hopefully this overview will provide some insight into the available 
methods so that readers can begin to consider useful combinations for 


their own multiple objective decision problems. 


Ibe! 





II. BASIC THEORY OF DYNAMIC PROGRAMMING APPROACH 
TO MULTICRITERION OPTIMIZATION PROBLEM 


A. INTRODUCTION 


This chapter is concerned with finding solutions to the following 
general problem, which will be called "MCO (Multicriterion Optimization) " 
problem. 


Find the set of nondominated solutions to 


maximize £, (x) 
£. (x) 
£ fx) 
subject to 
Ax <b 
x > 0 
whe re A is an M by N matrix 


b i Le geeer 
is a vector (b, -b, bi) 
x is a vector (X) X00 ++ 2%) 


£. (x) is a decomposable function 


The dynamic programming approach to the multiple objective function 
problem is of interest because we can decompose each objective function 
into N-stages in dynamic programming (if separability and monotonicity 
conditions hold). At each stage, we get the L single-variable return 
functions and we can find the nondominated policy by just eliminating 
the dominated policy. 


14 





Once we get the optimal returns (nondominated solution) from each 
stage, then we can easily obtain a nondominated solution for a system 
by Bellman's "principle of optimality" stated more succinctly in his 


words - 


An optimal policy has the property that whatever 

the initial state and decision are, the remaining 
decisions must constitute an optimal policy with 

regard to the state resulting from the first 


decision [20]. 


B. NOTATION 


The following notation and basic definitions will be employed in the 


remainder of this chapter. 


1. Multistage Problem Solving 


When solving a complex problem, we often break it into a series 
of smaller problems - decomposition - and then combine the results from 
the solution of the whole problem - composition. We call this approach 


multistage problem solving. 





[Figure 1] 


ILS} 





2. State (x) 


The state of each stage is defined by a set of numbers called 


state variables 


Oe ye yette %y 


We use a subscript n to denote the value of the state variables after 


stage n - see (Fig. 1). 
3. Decision Variable (D) 


The decision variables are those quantities that can be con- 


trolled or chosen in the design and operation of any stage 
D= (D,, Doreees Direecs Di) 
See (Fig. 1). 
4. Stage Transformation 


Each stage of the process transforms the state of its input into 
an output state in a way dependent on the decision that has been made 


for the operation of the stage. We express this symbolically by writing 


=t 
onal a De 


meaning that given x and dD it is possible to calculate Xo-1° 


5. Multiple Return Function 


Each stage has L return functions. We call this a "multiple 


return function" and the notation is as follows 


16 





eee FY 


[ 


where 
Mom i,2,..., N (stages) 


e=— ol 2 eG (number of return functions) 


6. Multiple Aggregate Return Function 


The total return R of the gan return from stage one through N 


is some function of the pee individual stage returns written as 


2 2 2 2 2 
Nene? Fa) 


Q=1,2,..., L (number of return function) 


7. Inequality Notation: 


If the objective functions of an optimization problem is vector 


valued, such that 
— ae Z | EE 
OP AOn nO pee, Oo. | r 


we define a partial ordering of elements Q as follows: 


ee 





Om2o) G0) <0 oP 
9°<g om oe ona ‘, 


Oo | pan 9° < ot +, 
ae = 5 Sale cameo ye 


8. Notation of Nondominated Points 


Given a set of feasible points 


W © ie 


anda VeECtor Criterion function 


then the set of nondominated points is 
1 
Weer W) REN & ete) TS en sige 
9. Notation of Nondominated Values 


Q={Z=R z= £(x), xen} 


The set of all nondominated solutions in the criteria space TS) will 


be denoted by 


where T.(X.) = ig(z (XD), Dyn Xyep Pyeg) 777 Fy (XP) ¥ Duress sD} 
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chat.1S, aaCe) is the set of all feasible values of the vector 


g(r (XD) xr (Xo Dy) 777 Fy OGD) Lona LE Die age! 


N-1 pe 


Subject to 


me = t(x ,D), n =1,2,---N 
n-l nen on 


and g(r (XD), ri Xyey Pye) 777 r, (X),D,)) is 


3 


the multiple aggregate return function. 


The optimal returm (nondominated solution) from N stages is defined as 
OS) Q [T.(X) ] 
subject to 

xX -1 = bP? n = 1,2,---N. 


10. Multiple Recursion Equation 


Assuming that the optimal retum, 


r 
<O| 


Fy (Sy) LT (XD) | is decomposable, 


then 


T 
© 
mama | 
| 
Pa 

8 
heed 


POX) 
where 


S(%) = (9, (er (XD) Fyiy (X40) * DY) 


aS) 





ehnat is, S.(X) is the set of all feasible values of the vector 


g, (ry (QDs Q (eee Oe, for all values of D.. 


This notation means that each Ly (Ky Dy) is combined with each vector in 


the set 2 [Ty (XJ 


subject to 


and let 


BD) = Go (= (EA eae en Clay 


then 


SX) {Q(X Dy) ¥ DJ}. 


Applying the same decomposition to Pogny (Xyiy) 777 Pi (X), we obtain the 


multiple recursion equations. 


Pea e [s_(x,) ] n = 1,---,N 


D 
n 


where 


Yn) 
= 
Y 
eal 
I 


fOCXe Deep. t 
| Is eae @ | n 


tO 
5 
> 
= 
i 


r(X ,D) n=l 
nm fh nN 


T 
H 
Oo 
O 
i 
> 
= 
= 
| 
NO 
{ 
( 
i 
Zz 


and “o" is the composition operator 
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subject to 


C. BASIC CONCEPTS OF "NONDOMINATED" 


Let us briefly outline the basic concept. We will utilize the 
Gecision variable Dd. as in Figure 2. The value of decision variable DS 


results in the value of Q, (XD) through the equation. 





Q(X, .D) = OK DL) Pn) 

D _ (X_,D) 
n SE AGC : 

(X_,D ) 


e, 
I 


1,2,...,N (number of stages) 
2=1,2,...,L (number of objective functions) 
X = fixed. 

n 


[Figure 2]. The relationship between decision variables and return 
functions. 


Each choice of decision values which is allowable yields a feasible 


solution in criteria space as in Figure 3a. 


1 2 
Q, (%)D,*) + Q, (%-D,*) 










1k, 1 2 ah 
on (XD, yy oh (XD, 


dominated 


[Figure 3a]. Feasible solutions. 


AA. 





The full set of allowable solutions obtained by mapping all allow- 
able values of the decision variable Do into the criteria space yields 
some volume in that space, again as in Figure 3a. 

In general, the majority of these feasible solutions will be domi- 
nated, that is, a feasible solution will exist which is at least as good 
in all criteria and better in at least one criterion. 

A "nondominated" policy can be defined in the following way. A 
point D* E x is said to be a nondominated policy if and only if there 


is no other policy a E Xx such that 
9 * (x 7D ) > Q tex PD weenetral ). 
Pon 2= =A non 
and 
OeeeserD)) 7 9,7(X,D.*), at least one i 
where x is fixed. 


That is, a "nondominated" policy has the property that there is no other 
feasible policy which would improve at least one goal variable while the 
others stay unchanged. 

The surface of nondominated solutions in criteria space for the 


example shown is indicated in Figure 3b. 


1 
Q 


jg | 


[Figure 3b]. Surface of nondominated solutions. 
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A set of representative nondominated solutions for the example shown 


is indicated in Figure 3c. 





[Figure 3c]. Representative nondominated solutions. 


D. BASIC THEOREM 
From the general M C O problem, 


maximize F (x) 


Sug OC CaO) ss Oem FOL ie ly, 2,7... ~ 10 


where x is an N-vector and F(x) a function of x, is an L-vector with 
scalar component (f, (x), £.(x),---,£, (x)) 
To use the dynamic programming method each of the L objective 


functions must satisfy the definition [1] and [2]. 
1. Definition {1} (Separability) 
Let the aggregate return be 
R= glx rj]. 
"Separability"™ can be written 


Sigs aeptre ocnca 


= Gy [eye Fy (yi yee seek) ] 
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whe re Jy and g, are real-valued functions. 


2. Definition |2| (Monotonicity) 


From g, [ty Go (Ky lpee ert) | 
oe is a monotonically nondecreasing function of I. for every fixed Oy 


If each objective function satisfies the definition [1], [2], then it 


can be decomposed into N stages. At each stage we get L retum 


functions. 
1 
r (xX_,D) 
— 2 
r (X 7p?) = 16 (X ep) 
nm in on n n on 
ne a 
n nm on 
n= 1,2,...,N (stages) 
Q = 1,2,...,L (number of return functions) 
and 
Q(X .D) =F (xX ,D) oF _,(t (X-D,)) 
giving 





We assume that there is some overall utility function implicitly 


defined on the Q.(% Dd). At stage M x is fixed and the feasible 


24 





values of D_ depend on X_. 
n n 
Hig carb.) |= ulo tix D_),---Q Hoan ) 
een nh ee erie tl heen on 


This function is unknown but it can be assumed to be real-valued, 
, y 
monotonic and nondecreasing in each argument On (xX -D)- 
The problem is to find D,* for fixed Xx such that 
= Ds . 
MAX u[Q (xD) | u[Q. (x ,D, ie 
D EX 
Gm hi 
3. Theorem [1] [Geoffrion | 
Let the set of feasible Dd, be compact. Then at least one policy 
at which u[O (xD) ] achieves its maximum over Xx (fixed) is "non- 
dominated." That is 
= _ a= és 
MAX ula (x -D) J u[Q (xD, ) | 
Deen 
igh 1 tal 
where Dest and {} = the set of nondominated points. 
PROOF 
By the compactness (bounded and closed) of the feasible set of 


dD, and the continuity of U there is at least one maximal solution to 


ay se ° e ? 
u[Q (xD) ], say D.*e2. Consider some point D ‘eX, such that 


aE | st we 5 = 
Q, (XD) 29 (XD *) i = 1,2,...,L 
ie 


and which maximizes £ Q * (x ,D_) over X_. The point D_' 1s obviously 
i=7 7 Nn on n n 


nondominated since otherwise there would be some Dex, with 


ie — at : 
OMe Dae > © 8 (%.,D.') for at least one i 
ak agi wae) nan on 
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and 
Kix Do) = 0 Kx D.') for the remaining k # i 
ae n “n= “=n no an J 
Then, of course, 
=O ae Do) Xe) = ,D_') which contradicts the choice 
n le" 501 mM on 
a: D '. 
n 
Since Dd. is nondominated 
> | =< +O Ba q 
Ont Di) ea) Ox, DB. ”) Q(X 1D.) Q(X, Dd, ) 
¥D eX 
n n 
Because U is monotone increasing, then also, letting Dd. = De 


Q(X -D.*) > Q(X ,D.') = >0,(X% -D,*) =O (X,-D,") 


implies 


u[Q (x ,D.*)] > Ufo (x .D.")] 


>ufo (x ,D * 
nn’ on "pli e ? mag! u[Q,, ( ne on ) | 


ufo (xD) 
SO, Do also solves 
MAX U[Q (X_,D.)J = ul (X.D,")] 


Dimex  . 
i an) 


Thus, it is shown that at least one policy at which U[Q (XD) J 
achieves its maximum over Xx is nondominated 

OnE. Ds 
The theorem provides the justification for solving for nondominated 


solutions. 
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Next, let us briefly explain the theory of the dynamic pro- 
gramming for the MCO problem. In dynamic programming the N-variable 
problems must be broken into N one-variable problems. So our objective 
is to decompose the problem 
Pe Ce = 2 [T,X J 


Nel 


where T.(X.) = {g(x (%,D.) , ony Xyep Pyeng) TOT (Xp Dy), se as la 


that is, TC) is the set of all feasible values of 


g(r (4 Dy), Dany Xyey Pye) 777 Fy (Xp Py)? for all D.,---D, 
subject to 


x1 = t (kD). n=L,2,-~-,N 


into N equivalent subproblems each containing only one state variable 
and one decision variable. Then the solutions from the subproblems are 
combined to obtain the solution to the original problem. To achieve 
this decomposition, a highly restrictive assumption must be made about 
the function g;- That is, a sufficient condition for decomposition is 
that 5 must be a monotonically nondecreasing function and another con- 
dition for the decomposition is the separability. 

In the case of a single objective function, a sufficient con- 
dition for achieving the decomposition has been given by Mitten [33]. 
We will prove that a condition similar to Mitten's is a sufficient 
condition for the case of multiple objective function in theorem [2], 
ier [4]. 


To decompose the problem 


BN 2 [T (4) ] 





where Thy Sy? = tg (x (XD), ( 


Reo ane weet oye © Pyr-—-Dy} 


we must achieve the crucial step of moving the optimization (here finding 


--,D, inside the Nth 


all nondominated solutions) with respect to D l 


wee 7 


stage return. Let the multiple objective function be separable as 


follows: 


of COEF) BE ae (XD 


NaN chy N-1 ),--- x, (X%, -D,)) 


N-1 
en ey es ee eae ete pny 2? 


then 


=e = = “ _ 
tg(r (XD se My (yy Dy yg) r,(X).D))),¥ Di D, } 


TOS) 


{g, (rn (XD) Go (Oy (Xi  Pyi) 7 ey (XD) 


Die Day 


From the definition of F(X) it follows that 


Poy yey? = 2 [Ty (Xyey) J 


whe re 


Tye Sye1) = 92 yey Rye Pyeg) 777 Fy (Xp DY) F Demon Dy - 


Then the decomposition is as follows: 


Fy (%) LS (X) J 


N 


where 


Sy (Ky) = tg, Oy (XD Fey Oy) F Dd 


= (9, (r (XD. Fyiy tylXyeDy) ¥ DYt- 
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n-1' 


It is clear that we can proceed further by treating ) and then 


yea 

ieee (X_ 5) ---F 5 (X,) in the same way as F(X) - We decompose the 

original problem into N one-stage initial state optimization problems. 
1. F,(X)) = 2 [ 1%) J 


=k 


: where S, (X)) {r) (x, .D,), ¥D,} 


n. F (xX) = [S_(X_)] 
n n Nn n 


D 
e n 


‘ where S{%) {g(x (X_,D) v Foz ft, (8-2) )) ’ We DS 


So RR.) oa [SX J 
N 


{g(x (X..D), Fy, Ct, (XD, 


where Sy (Xp) Vy ys nee) 


Stating the N problems more compactly, we have 


exe) = 2 [Ss (x), n = 1,2,---,N 


D 
n 


where S_(X_) = {g(x (X,,D,), F__j(t,(%,-D))), ¥D} 


i 


{Q(x -D). ¥ Di} 


and Q (XD) = x (xX ,D_} fe= ok 


Cre) Eom eine) en 23, ———,N 


This equation represents the usual recursion equation of dynamic program- 
ming, but for our application the usual max or min is replaced by the 


Operator 20. 
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We are now in a position to prove the decomposition in the case 


of multiple objective function. Given the separability and monotonicity 


of g; we must prove 


es eo = 1S 
D_,---,D 
1 
where 
Ty OX) = 9, (x (XDD) ’ J5 (ry Sy Py_d? 177X, (KX, Dy) er 


ey Dyeee Dy 


Sy (Sy) = 195 (Ey OGD) Fey (Oy (Sq DQ) F DSI. 


For the convenience of proving the decomposition, we transform 


the multiple aggregate function, 


Rye My igs W%yr Dye Dy) £0 ROyeDy y-—-/Dy) 


by using the stage transformation 


cee Eee Ds) 


The stage multiple return is 


mr yo 


From the transformation, it follows that Xx depends only on the decisions 


made prior to stage n (D a) and Xe that is, 


+1’ 


an e tei! 


) 


Sa OP na ~ Ce ong 2 neo nee 


=i ( 


_ eee (Ee a 


X 
ee re re n+l n+3 eee oats 
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It then follows, by combining the above equation with the return 
function, that the return from stage n depends only on the decisions 


[> ,D 


S mes 1D) and Xa that is, 


aes = xX Dy) = Deng Sy Oy 777 P ), D) 


I 


rf Xye Dye tro DL) - 


The multiple aggregate return function R from Stages one through N is 


some function of the individual stage returns written as 


Se et ane oN Sno oe Sg oy! ON 


Na Ne aNet) weeny)? - 


However, as just explained, (X77 Ry) can be eliminated from the 
individual stage returns and consequently from the total return. Thus 


an alternate expression for R is 
Eon ame! eS Sree On ge Sy Pg P yi) eo 


(Xp Dye-o- Dy) 


Let the multiple aggregate return function be separable as 


follows 


ct eee Gn Ox Do) gy (m4 (XD po) e--—, 


N 


Vy (Xe Dyer Dy) )) 


and ory be monotonic nondecreasing function of Io ¥ fixed ry 


where 
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Let 


where 


Th (X) = {g, (x (XD), Big a ry Py Py? ie 


Xe rDue Dy 1! a —1D)))) Pe i Demo eD, 


Bx.) is the set of all feasible values of 


Fy (Ty (KyrDy) e Gy (Tyg (Kyi y Pye Dy ag) oom 
ry (X Dyer Dy) )) 


Boe all D _,D -<--,D 


N’ N-L’ a 


and 2 [T(x ] is the nondominated solutions among the values in 


* in 


the set OF Ty Ay) 


Let F! 4) = Q [s. (X ) | 


where 


Sy(%y) = gy (ry KyrDQ) Q [Ty yy) De # DY) 


SZ 





we must prove 

ee oN 
For notational simplification let 
Fe (X,) 
AON 


{w} 


4); 


Note a Ze Bsc) is nondominated in comparison with all other feasible 
values, while awe Ea (5) is nondominated in comparison with the values 


enumerated in the evaluation of Q [s(x]. 
D N 


4. Theorem [2 ] 


{w} c {z} 


PROOF 
Suppose gw such that wc {w} 


then from we {wi} = Eee 


for some De * ,---,D,* 


= me Ge * = * * Sa 
y 9 bry (ye Dy ys Go (Fy ye Dy Ie els / 


=) CoB oa) Ne 


Then it is false that there exist Da De jena :D, such that 


~~ 
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for any Dee =e Since w is nondominated. 


But if w ¢ {z}, this cannot be for either of two reasons: 


-,D 


(1) either w is dominated in {z}, then there exist Dg Big ao 1 


such that 
g, (x D g.(r. 5) pI --- ---,D > w. 
J, (Hy OY Dy) Jo (Yyy Xye MH Reg! 6 x, (X, (Der 7D,))) 2w 


This contradicts the assumption w « {w}. 


(2) or else w £ {z} can occur because w was not enumerated when 
{z} was computed, 


but this cannot be because 
{tw} c fg) (4 (XD) se G5 (ty (Xe Dye Dy) econ ig y XygeDyeroe Dy) )) s 


¥ Di--- sD, 


that is, 


whe {T.(x,)}. 


This can be proved by the following proof 


We add Ky (XD) to both sides, then 


{g, (x, (X/Dy) ’ 7) [To ED }) } Cc {g, (4 (XD) ’ 


Ter %y-v))} 


By monotonicity of 9) this equation is true. 


And this can be expressed as follows using the definition of Sy (Xp) 


(X) 
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{s(x F ¢ {T(x 3 


and ad [S.(X) J c {8 (X.) } 
thus - moe chomp ec 1t7(x7)) 
N 


ene ee oe 


So w was enumerated when {z} was computed. Thus if W « {w}, we also 


have we {z}. 


5. Theorem ED 
{z} c {w} 


PROOF 


Suppose there exist z such that z e« {z} 


(1) Then from 


ZAG Z) = Q a (X tl 
Dorma rDy 

where 

pee) Magy (e (X 7D), go(z (XD. :~—=, 

£y (Ky rDyeroo Dy) 4 # Dy_yeo-Dy] 

for some Do*,---D,* 

Z = pecan >. *)), eae eee eae meet OND ree) D4)*))))) 
and for all Das Dae 1! —--,D,- 


a5 





The following equality is false: 
By (ag Eko Du) + Ge 1 Xye Doe OO oie 
ie =a > 
r, (4 .Do Da Mia z 


, 7 * 
(2) Also if we f1x Xo Dy , then 


gy (x, 


k ee eee ek ---p * 
N- ~1 Sy Da Dv-1 ie 10) (XH Dy ’ D. )) 


a = 5 [Ty-7 %y-y) | 
N-1’ ae kl 
whe re 
Tyay Xyez) = £99 y- Xye Py Py) —1Xy (NyrDy emo Dy) 
aDemnesaay® } 
Since otherwise its domination (say by, 
Fo (Tyg (Kye Dyy* + a) Oe Tye ye Dy ~1Dyig) oo : 


ry (X,,D.*,---,D,)))) 


would imply domination of z by 


Jy (x (KD ne (8 _1 ye De py) 777 Fy CRD oo Dy) )) 


(by monotonicity of g,) imolwinaeze er 1254 
And now suppose z ¢ {w}, this means either 
(a) zwas not enumerated when {w} was computed. But this 


cannot be by (2), or 


~ 


(b) zwas dominated in {w}, then there is some Digr--= 1D, 


combination such that 
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Gy (4 (Xe Dy) « Sig (Reo WOK GDh) aca 
r Del ao > 
ry (Xe DyeDy ye 7D,))) i Zy 


but this contradicts (l). 


Thus if z ¢« {z}, we also have z ec {w}. 


6. Theorem [4] 


{w} = {z} 


PROOF 
By Theorems [2] and [3]. 
From Theorems [2], [3] [4], we have shown that if Fy is a monotonically 


nondecreasing function of go for every fae then 


where 
Ty (X) = tg, (4X -Dy) ' Pye Xy-1 One? p77 Ty (KX, -D,)) ’ 


yp Diyr7 77 Dy} 


Sy (X) = tg, (4 04) DD Z ae [Toy (My) Ds ¥ DY 
N-1’ eel: 


Thus, the stage by stage dynamic programming recursion will generate 
exactly the set of all nondominated solutions to the original multiple 


Criterion optimization problem. 


ay) 





IIIT. BASIC COMPUTATION 


A. GENERAL SCHEME (DISCRETE CASE) 


Computational aspects of dynamic programming approach to MCO concern 
the solution of the multiple recursion equation. In the last chapter we 


defined the multiple recursion equation as follows. 


ee) 7 [s_ (x) ] (1) 
n 
where 
s{%,) = {Q(X Di). y Di} 
Q(X .D) - ri {%/DL) n=1 (2) 
= x (&)-D_) O Finitty X_Pp?? (3) 
Mec 2 pe ete NN 
with X-1 = t(D). 


First, we start with n = 1 (Stage 1) calculating 


i 

1 §% 7 Py) 
2 
1 


1g 


(X AJB, 


17?) 


r, (x) /D,) = ie 1 


(X),D,) 


Since n= 1. From Equation (2), 


Q, (x, ,D)) = r, (xX) Dy) 5 
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Equation (1) is then used to obtain 1 Sy) and Dy (X)) by using the non- 


dominated method. These are saved for future calculations. But Q and 


r, are no longer needed. Since n # N, we increase n by 1 and calculate 


r, (X,,D,) 


for n = 2. Q, is calculated from Equation (3) by appropriately combining 


= and in Py has now served its purpose and may be discarded. 

The calculations continue similarly for n = 3,...,N. Thus we obtain 
Bex), the nondominated solutions from the N-stage system, and D(X), 
(n = 1,...,N), the optimal nth-stage decision function of the nth-stage 
inputs. 

The next step is to determine the optimal inputs x" healer N—1) 
and the optimal decision D* (n = 1,...,N). We have reached the point in 
the flow chart (see Fig. 1) where, for the first time, the answer to the 
question "Does n = N?" is "yes." 

We assume there is a prescribed value of x denoted by xe D. is 
obtained immediately from the stored function D(X) - Then Xt is 
calculated from X* 1 = ty (* Dy) - Since n # 1, we compute Det from 


X and the stored function Oke) When the answer to the 


x 
N-1 


question "Does n = 1?" is "yes," the optimal solution is obtained. 
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Compute 
i @ (X 7D ) 
gy Sgt Heel 
















(X 


© (X% ,D jer (X ,D )oF 
ao oa n a ssi ne on n-1l 











{ ———-) — i =as was mcs) sma 
& _ 1 
ee Seve ra) 
nee nh i 
} [a eS a eee 
—_—_—— Seo eee 
| 7 
oe ISAVE D (X ) | 


Yes 


Solution C 


[Figure 1}. Flowchart of General Computation 
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B. TABULAR COMPUTATION 


The term tabular computations is used to describe dynamic programming 
analyses in which the optimal decision functions and optimal return 
functions are given in discrete form in lists or tables. In other words, 
for each feasible value of Xue n=1,...,N there is an entry in a table 
containing values of De (x) and BOS Ne (See Table 1.) 

Naturally, it is only possible to give this information in a table 
when there are a finite number of feasible values for Xo But tabular 
representation of continuous data is possible if we make discrete 
approximations. For example, suppose the feasible values of x are the 
integer numbers from 1 to Ka For x = K (=1,...,K)). D* (X SK) is the 


set of all Do which are nondominated points of 
Q (X =K,D.) = r (X =K,D.) o F. .(t (X =K,D_)) 
ne aon n eh n n-l nin n 


and FL (X=) is the nondominated solution value of the above expression. 


At the first stage, the feasible values of X, are the integer numbers 


uf 
Brom 1 to K,: Suppose that for each feasible value of X,(0,1,2,..-,K), 


the feasible decision variable value is D, = 0,l,...,X 


1 (See Table 1.) 


1: 
For each value of Xper (X)/D,) is calculated when Xj =K. The multiple 


return functions are 


1 2 L 
r,(K,0), r,(K,0) ,---, r, (K,0) 
1 2 le 
r, (k,l), rv, (K,1),---, r, (K,1) 
1 2 L 
r,(K,K), r,(K,K) ,---, r, (K,K) 
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[(t) s] 8 : 
u u u u 
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and Dy is a nondominated point if there does not exist another Dy such 
that 
i ‘ x 
tix D.) x elie D, ) ge ib t one i 
va ea teoeeeal on 
— { 
Dy em ee yao op ik DS a Dy 
dD, 00 pi eee. 
that is 
F,(X)) = @ [S, (x))] 
where 
S(X,) = {Q,(X,,D)) ¥ DJ 


Q, (x, Dy) = r, (X,D,). 


The tables of BL GS) and D, (X,) are saved for future calculations. 


(See Table 2.) 
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(X= xX) a (X= xX) a ~ =n x 
= ¥ 
Uu Uu Uu Uu Uu 
(1 = xX) d OT od 4 = X 
Uu Uu Uu Uu Uu 
(Oo xX) a (G@=) peed QO = xX 
s ¥ 
Uu Uu Uu Uu 
i (xe) a 
uoTINTOS STqCetxeA UOTSTIAG :: 
pozeutTWwopuOoN paezyeuTWOPpUON X oTqetzean 3309S AOZJ 
pewzetToossy poezyetoossy SONTPA OTATseeq Jo 3ST 





(A4TGVL AUYVWWNS) 


c¢ ATaAVS 
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Finally, suppose that at the last stage (N), the feasible value of 
the state variable is just one (K.) and the feasible values of decision 
variable are 0 through Kae and every entry in the table is the same as 
in Table 1. The calculations are basically done in two parts; first, 

° * * * 
--- X 
the calculation of Py (Xd) and D(X), D,(X,), F Df “i and the second, 
* 


the tracing of the nondominated policy Darn Dy - 


At the last stage we get on = Ko and ore then by transformation 


X . eC ae we can compute x” and by the summary table of 


N N-1 
as _ ae i * 

Stage N-l (we saved each Pyar Syn)? a Ky Dy_y (Xy_1)) We can 

compute ene Care We will repeat this until n=l; then at least we will 


get D*, D* _,---,D 


N N-L’ 1 


C. AN EXAMPLE: ONE STATE VARIABLE AND IRREGULAR RETURNS AND TWO 

OBJECTIVE FUNCTION 

This example shows how tabular computations are organized and 
provides some clues to the number of computations involved when the 
recursion equations are solved by enumeration. 

Example Problem. 

Five OA students meet to work the homework problems in two courses, 
D.P. and Stochastic Models; both have three problems. They agree to 
split into three teams...each team assigned one D.P. problem and one 
stochastic problem. The probability of success on a problem depends on 
the number of students working on it. (No student can work on more than 


one team. Assume all students' capabilities are equal.) 
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Probability 






Number 
of Students 





Stochastic Project 


NGEEe Probability 





How many students should be on each team to maximize the joint proba- 
bility that all problems in D.P. are solved and the joint probability 
that all in Stochastic are solved? (At least one student must be on 
each team.) The formulation of problem. 


3 1 
Maximize 2 AD.) 
n n 
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tl wm Ww 
wy) 
| A 
Ww 


Subject to 


O 
Iv 


ae) 
i 


where 


to 
I 


~ 
ll 


Multiple recursion equations 


x) - ge oe where ce) 
F(X) = Q [S,(x,) ], where S, (X,) 
Q, (X,) 
x.) = = [S,(X,) ], where S,(X.) 

3 — 
Q,(X,) 

BS (pt, Belle 

Computations 
Stage l 


OF Do = integer 


D P probability 


stochastic probability 


{P) (xX, ,D)), 
{Q,(X,,D,), 
P. (X,,D,) O 


{9,(X,,D3), 


P(X, D5) re) 


1,2,3(number of problems) 
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Summary Table l 












S F(X5) Pa CSe 
2 1 
1 
2 
1 
2 
3 
0. (,D) 
0.054 0.018 


0.048 0.024 
020272 0,.025 


0.060 0.064 | 0.060 0.064 






0.040 0.084 
020367 0.070 


0.040 0.084 
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By back tracking we find the nondominated solution to contain two parts 


imply: 
(1) D, = 2 D.P. = 0.06 
D, = aE Stochastic = 0.064 
= 2 
me 


Cp. = £ Pee — O04 


a 
D. = 2 Stochastic = 0.084 
D, = 2 
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Stochastic 
? (es Aes, 


ORO) 7 le 23) 


(A) 


0.06 


2.03 Glacieale 


F254) 


(3,150) 


migure 2, Nondominated solutions in two-dimensional 
Gritcerta Space. 





D. THE CONTINUOUS VARIABLE CASE 


Let us consider the problems involved in solving MCO problem when 
we no longer require that the Pe be integer. We shall then also drop 


the restriction that ar ; Kk be integers. That is 


| 
D 
‘a 
= 
A 
| 


F(X) 


where 
Ty (Xy) = tg (xr. (X eDy) ’ iy Xe Pye y? p77 Ty (KX, D,)) ’ 


} 


Lee 


In carrying out the optimization we consider only non-negative oe which 


satisfy 


Precisely as before, 


Q [S_ (x) ] 


F, (X.) 
laraea a eae i 
= < 
o<D, <K/a, 


where 
S,(X)) = {x, (X,-D,), V D, }. 


Note that D, varies up to K /a, not [kK _/a,]- The same recurrence 
relations follow that applied to the case where the Be were to be integer. 


That is 


F (x) = Q [s_(x_)] 
igi 1 1) o<D <K /a nm on 
cei) @ aia @ | nN 
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where 
S_(%,) = 10, (% Db), ¥ DI, 
ai) 7 Soy - a eden 


In order to apply the dynamic programming computation using the 
tabular form, the decision variables must be discrete. So first we 
must decide on a discrete grid to approximate the continuous variables. 
If the grid size is A, the length of interval (L) then the number of 


points (p) on which the function is defined is 


Thus for the fixed L, p is inversely proportional to A. If p is too 
large, then the D.P. is almost impossible to calculate, we must reduce 
the size by making the grid coarser. That is, by changing the grid from 
A to a larger value 6 there is a corresponding decrease in p. 

To see how the computations decrease as the grid increases, suppose 
we want the optimization of Q(X +d) for fixed Xe and o<D_ $20,000 on 
the grid qd. = 0.1,---,20,000. If we start initial grid, there are 
20,001 points at which the function is to be evaluated. 

By increasing the initial grid size to d = 0, 100, 200,---,20,000 
and evaluating the function at 201 points, the interval of optimality is 
reduced to a length of 200. Then another 201 evaluations of the function 
will yield the integer solution. But the number of computations can be 
reduced still further by starting with a very coarse grid d = 0, 1000, 
2000,---, 20,000. Then the interval of optimality is reduced to a length 
of 20. Then another 21 evaluations of the function will yield the inte- 


ger solution. 


3) 








Assume that the function a (xX _1) is known, then On is defined 


-1 
over a continuum of values Dd. in the interval o<D <K /a,- 
1 
1 = j lotted (as s (XD ) vs 2 (7) s) 
2, see aa © mone on Q ee on 


(at fixed x) 


it might look like a space or a curve. (See Figure 1.) 


[Figure 1] 


And the nondominated solution set is the thick line. We would like to 
determine the set of values D(X) at which oO assumes its nondominated 
solution. To reduce as much as possible the computational effort 
required, one might do a rough initial search using a coarse grid of 


Dae (D, = 0,5,25,---D ). (See Figure 2.) 


5 





@D @ = 


OC OF 





[Figure 2] 


From this coarse grid of decision variables an approximate nondominated 
policy would be found from Figure 2, points A and E. Then we can 


conclude that 


arama sedan es do% 6 


that is, QO < dA*n < 6 


36 aA tn Seer 


A finer grid would be used in the neighborhood of the nondominated 
point to determine more nondominated values or to locate them more 
accurately. The procedure of using a finer grid might be repeated 


several times to obtain the desired number of nondominated points in 





D(x). 
i 
Qn 
ee ea See ae a ee ee Wa Ve 
A B e D E F 
[Figure 3] 
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Consider next the problems which arise in the tabulation of the 
CX ) - A difficulty is encountered here because F(x) is defined for 
all Xue O<xX_<K_- Furthermore, examination of the search procedure 
introduced above shows that F(x) may actually be needed for any x in 
this interval, due to the fact that when finer and finer grids are used 


to determine F (X_) for a givens, F 
non n 


n-1 *n-1? may be required fora 


very large number of different arguments. It would usually be impossible 
to tabulate the F(X) for all possible Xo that might be required with 
the search procedure described above. Clearly, if this cannot be done, 
we must tabulate the Ea (xan) for a smaller number of Xo: 

In dynamic programming the main benefit of a coarse grid on the 
decision variable is that it produces a corresponding coarse grid on 
state variables. 


In the relation 


oneal ~ Ey Xe Dy) 


the number of feasible values of x is roughly proportional to the 


= 
number of feasible values of D_- By reducing the number of feasible 


values of Xo-L! we evaluate BL (X41) fewer times and the computational 


=). 
procedure is sames as discrete case. The successive grids used ina 


dynamic programming analysis depend on the number of values tolerable for 


the state variables and the accuracy required. 


E. COMPUTATIONAL EFFICIENCY OF THE METHOD 


Let us now compare the computational efficiency of the dynamic pro- 
gramming approach to solving the MCO problem with that of simply 


enumerating all possible sets of non-negative integers pe which satisfy 


>> 





the constraint 


For illustrative purposes 


N 
let a. = 1 then £a.D.<K become 
J az j7 7 
j=1 
N 
x D.<Kn (1) 
j=. 7 


and only one constraint and one variable. 


L. Direct Enumeration 


In the case of N stages and L objective functions and the 
constraint is (1) then the number of evaluations of objective functions 


is 


and the number of comparison for choosing nondominated solutions is 


= +K— 
tn ee *) aa) (n K I, 
K K 
a 5 : 
K=0 
1 2 L 
This shows the number of comparison of vectors (f , f ,---,f). 


2. D.P. Approach 


At each stage, there are (Kn+l) x values and for each value of x 


there are (x+l) D values. The number of evaluation of objective functions 
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at one stage is 


Kn 
WE G1) = L((kntl) + Ko nti) 
x=0 


All (N-1) stages have the same number of enumerations 


Kn (Kn+1) 


L x (N-1) [Kn+1) + 5 


and last stage has just (Kn+l) x L computations. Thus 


Kn (Kn+1) 
Z 


[(N-1) ((Kn+1) + ) + (Kn+1)] x L 


and the number of comparisons for choosing the nondominated solutions is 
at each stage 


Kn 
& x(x+1) 
x=1 2 


and since all (N-1) stages are the same, the total number is 


Kn 
x (x+1) 
(N-l) 2 7 


=1 


Kn (Kn+1) 


At the last stage, the number of comparisons is 5 


thus the total number in N-1 stage is 


(N-1) y x (x+1) A Kn (Kn+1) 
x=1 2 2 


Therefore total number of comparisons for all stages is 


Kn (Kn+1) n Kn (Kn+1) 


(N-1) [$5 (Kn+1) (2Kn+1) + mn aa 
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For example (2 objective functions) 


DIRECT ENUMERATION 


10 50 100 1000 


jo 


3 RiesaG 3.5x10> 
5 1.9x10° 
10 : elo.” 5 .8x107" 





DYNAMIC PROGRAMMING 


N * 10 50 100 1000 


3 4 
5 2 86 5 .4x10 2 EX1O 2x10° 
5 550 1x10° 4.1x10" 4x10° 
10 1210 2.4x10" 9.2x10° 9x10° 


Notice that the direct enumerations grow exponentially and the dynamic 


programming approach increases linearly with N. 
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IV. CONCLUSIONS 


We have discussed the concept of the nondominated solutions and have 
shown how the concept of the dynamic programming can be applied to 
determine the nondominated point set. 

We have also discussed the sufficient conditions for the decomposi- 
tion of multiple criteria functions. Problems involving both the 


discrete and continuous case can be handled. 
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